Introduction

In Carola Lilienthal's (###TWITTERHANDLE###) talk about architecture and technical debt at Herbstcampus 2017 (###LINK TALK###), I was reminded that I wanted to implement some of the examples of her book "Long-lived software systems(available only in German) with jQAssistant. Especially the visualization of the dependencies between different business domains seems like a great starting point to try out some stuff:

The green connections between the modules show the downward dependencies to other modules and the red one the upward dependencies. This visualization can help you if you want to further modularize your system towards your business domains or subdomains.

At the same time, I started the Java Type Dependency Analysis and realized that there it is only a smart step to analyze dependencies between business domains. What's missing is the information which type belong to which business domain. We'll find out now!

Core idea

I claim that every software project's source code has a hierarchical structure. That means that each element in the software can be assigned to a thing of a higher abstraction level like "technical purpose" or "business domain". In most software projects, you'll be able to do that with naming conventions found in the code. In extreme cases, you have to map each software entity (a class, interface or enum for example) to a high level thing by hand.

jQAssistant takes the mapping of software entities to higher level abstractions one step further: jQAsissant not only provides the means to derive higher level abstraction from naming conventions, but also from structural information. This is a extreme powerful way to abstract all software entities in your software to concepts that software architects or even business expert can understand and thus reason about it!

An example: If we take a look at the structural definition of Spring Petclinic, we can see that jQAssistant can mark software entities (in this case Java types) depending on their relationship to other sofware entities.

[[spring-mvc:Controller]]
[source,cypher,role="concept"]
.Labels all types annotated with "org.springframework.stereotype.Controller" 
 with "Spring", "Component" and "Controller".
----
MATCH
  (controller:Type)-[:ANNOTATED_BY]->()-[:OF_TYPE]->(annotationType:Type)
WHERE
  annotationType.fqn = "org.springframework.stereotype.Controller"
SET
  controller:Spring:Component:Controller
RETURN
  controller as Controller
----

In the case above, jQAssistant assigns the labels Spring, Component and Controller to every software system that is annotated with the annotation's type org.springframework.stereotype.Controller. A Controller is a high level technical component.

You can also get information about your business domain if you just map


In [5]:
features = ['Controller', 'Repository', 'Monitoring', 'Entity', 'Formatter', 'Service', 'Mapper', 'Extractor', 'Initializer', 'Util', 'Validator']
business = ['Owner', 'Pet', 'Visit', 'Vet', 'Specialty', 'Clinic']

In [6]:
domaininfo = []
for fileinfo in json_data:
    fqn = fileinfo['name']
    name = fqn.split(".")[-1]
    
    info = {}
    info['fqn'] = fqn
    info['technical'] = fqn.split(".")[-2]
    
    for feature in business:
        if feature in name:
            info['business'] = feature
            break
        else:
            info['business'] = "Framework"
        
            
    for feature in features:
        if feature in name:
            info['functional'] = feature
            break
        else:
            info['functional'] = "Other"
            
            
    domaininfo.append(info)

domaininfo


Out[6]:
[{'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.PetclinicInitializer',
  'functional': 'Initializer',
  'technical': 'petclinic'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.repository.jpa.JpaPetRepositoryImpl',
  'functional': 'Repository',
  'technical': 'jpa'},
 {'business': 'Visit',
  'fqn': 'org.springframework.samples.petclinic.web.VisitControllerTests',
  'functional': 'Controller',
  'technical': 'web'},
 {'business': 'Vet',
  'fqn': 'org.springframework.samples.petclinic.repository.jdbc.JdbcVetRepositoryImpl',
  'functional': 'Repository',
  'technical': 'jdbc'},
 {'business': 'Framework',
  'fqn': 'org.springframework.samples.petclinic.web.CrashControllerTests',
  'functional': 'Controller',
  'technical': 'web'},
 {'business': 'Visit',
  'fqn': 'org.springframework.samples.petclinic.repository.jdbc.JdbcVisitRepositoryImpl',
  'functional': 'Repository',
  'technical': 'jdbc'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.web.PetTypeFormatterTests',
  'functional': 'Formatter',
  'technical': 'web'},
 {'business': 'Owner',
  'fqn': 'org.springframework.samples.petclinic.web.OwnerControllerTests',
  'functional': 'Controller',
  'technical': 'web'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.repository.springdatajpa.SpringDataPetRepository',
  'functional': 'Repository',
  'technical': 'springdatajpa'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.web.PetController',
  'functional': 'Controller',
  'technical': 'web'},
 {'business': 'Clinic',
  'fqn': 'org.springframework.samples.petclinic.service.ClinicServiceJpaTests',
  'functional': 'Service',
  'technical': 'service'},
 {'business': 'Clinic',
  'fqn': 'org.springframework.samples.petclinic.service.ClinicServiceJdbcTests',
  'functional': 'Service',
  'technical': 'service'},
 {'business': 'Visit',
  'fqn': 'org.springframework.samples.petclinic.model.Visit',
  'functional': 'Other',
  'technical': 'model'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.repository.jdbc.JdbcPet',
  'functional': 'Other',
  'technical': 'jdbc'},
 {'business': 'Framework',
  'fqn': 'org.springframework.samples.petclinic.util.CallMonitoringAspect',
  'functional': 'Monitoring',
  'technical': 'util'},
 {'business': 'Owner',
  'fqn': 'org.springframework.samples.petclinic.repository.jpa.JpaOwnerRepositoryImpl',
  'functional': 'Repository',
  'technical': 'jpa'},
 {'business': 'Owner',
  'fqn': 'org.springframework.samples.petclinic.model.Owner',
  'functional': 'Other',
  'technical': 'model'},
 {'business': 'Clinic',
  'fqn': 'org.springframework.samples.petclinic.service.ClinicServiceSpringDataJpaTests',
  'functional': 'Service',
  'technical': 'service'},
 {'business': 'Framework',
  'fqn': 'org.springframework.samples.petclinic.model.Person',
  'functional': 'Other',
  'technical': 'model'},
 {'business': 'Vet',
  'fqn': 'org.springframework.samples.petclinic.repository.jdbc.JdbcVetRepositoryImpl$1',
  'functional': 'Repository',
  'technical': 'jdbc'},
 {'business': 'Owner',
  'fqn': 'org.springframework.samples.petclinic.repository.OwnerRepository',
  'functional': 'Repository',
  'technical': 'repository'},
 {'business': 'Clinic',
  'fqn': 'org.springframework.samples.petclinic.service.ClinicService',
  'functional': 'Service',
  'technical': 'service'},
 {'business': 'Vet',
  'fqn': 'org.springframework.samples.petclinic.repository.jpa.JpaVetRepositoryImpl',
  'functional': 'Repository',
  'technical': 'jpa'},
 {'business': 'Specialty',
  'fqn': 'org.springframework.samples.petclinic.model.Specialty',
  'functional': 'Other',
  'technical': 'model'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.web.PetTypeFormatter',
  'functional': 'Formatter',
  'technical': 'web'},
 {'business': 'Framework',
  'fqn': 'org.springframework.samples.petclinic.web.CrashController',
  'functional': 'Controller',
  'technical': 'web'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.web.PetControllerTests',
  'functional': 'Controller',
  'technical': 'web'},
 {'business': 'Clinic',
  'fqn': 'org.springframework.samples.petclinic.service.AbstractClinicServiceTests',
  'functional': 'Service',
  'technical': 'service'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.web.PetValidator',
  'functional': 'Validator',
  'technical': 'web'},
 {'business': 'Framework',
  'fqn': 'org.springframework.samples.petclinic.model.ValidatorTests',
  'functional': 'Validator',
  'technical': 'model'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.web.PetTypeFormatterTests$2',
  'functional': 'Formatter',
  'technical': 'web'},
 {'business': 'Owner',
  'fqn': 'org.springframework.samples.petclinic.repository.jdbc.JdbcOwnerRepositoryImpl',
  'functional': 'Repository',
  'technical': 'jdbc'},
 {'business': 'Framework',
  'fqn': 'org.springframework.samples.petclinic.util.EntityUtils',
  'functional': 'Entity',
  'technical': 'util'},
 {'business': 'Visit',
  'fqn': 'org.springframework.samples.petclinic.repository.VisitRepository',
  'functional': 'Repository',
  'technical': 'repository'},
 {'business': 'Owner',
  'fqn': 'org.springframework.samples.petclinic.web.OwnerController',
  'functional': 'Controller',
  'technical': 'web'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.repository.jdbc.JdbcPetRowMapper',
  'functional': 'Mapper',
  'technical': 'jdbc'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.model.PetType',
  'functional': 'Other',
  'technical': 'model'},
 {'business': 'Owner',
  'fqn': 'org.springframework.samples.petclinic.repository.springdatajpa.SpringDataOwnerRepository',
  'functional': 'Repository',
  'technical': 'springdatajpa'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.repository.jdbc.JdbcPetVisitExtractor',
  'functional': 'Extractor',
  'technical': 'jdbc'},
 {'business': 'Clinic',
  'fqn': 'org.springframework.samples.petclinic.service.ClinicServiceImpl',
  'functional': 'Service',
  'technical': 'service'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.repository.jdbc.JdbcPetRepositoryImpl',
  'functional': 'Repository',
  'technical': 'jdbc'},
 {'business': 'Vet',
  'fqn': 'org.springframework.samples.petclinic.repository.VetRepository',
  'functional': 'Repository',
  'technical': 'repository'},
 {'business': 'Vet',
  'fqn': 'org.springframework.samples.petclinic.repository.springdatajpa.SpringDataVetRepository',
  'functional': 'Repository',
  'technical': 'springdatajpa'},
 {'business': 'Visit',
  'fqn': 'org.springframework.samples.petclinic.repository.jdbc.JdbcVisitRowMapper',
  'functional': 'Mapper',
  'technical': 'jdbc'},
 {'business': 'Framework',
  'fqn': 'org.springframework.samples.petclinic.model.NamedEntity',
  'functional': 'Entity',
  'technical': 'model'},
 {'business': 'Vet',
  'fqn': 'org.springframework.samples.petclinic.web.VetController',
  'functional': 'Controller',
  'technical': 'web'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.model.Pet',
  'functional': 'Other',
  'technical': 'model'},
 {'business': 'Vet',
  'fqn': 'org.springframework.samples.petclinic.model.Vet',
  'functional': 'Other',
  'technical': 'model'},
 {'business': 'Visit',
  'fqn': 'org.springframework.samples.petclinic.repository.jpa.JpaVisitRepositoryImpl',
  'functional': 'Repository',
  'technical': 'jpa'},
 {'business': 'Vet',
  'fqn': 'org.springframework.samples.petclinic.model.Vets',
  'functional': 'Other',
  'technical': 'model'},
 {'business': 'Visit',
  'fqn': 'org.springframework.samples.petclinic.web.VisitController',
  'functional': 'Controller',
  'technical': 'web'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.repository.PetRepository',
  'functional': 'Repository',
  'technical': 'repository'},
 {'business': 'Vet',
  'fqn': 'org.springframework.samples.petclinic.web.VetControllerTests',
  'functional': 'Controller',
  'technical': 'web'},
 {'business': 'Visit',
  'fqn': 'org.springframework.samples.petclinic.repository.springdatajpa.SpringDataVisitRepository',
  'functional': 'Repository',
  'technical': 'springdatajpa'},
 {'business': 'Framework',
  'fqn': 'org.springframework.samples.petclinic.model.BaseEntity',
  'functional': 'Entity',
  'technical': 'model'},
 {'business': 'Pet',
  'fqn': 'org.springframework.samples.petclinic.web.PetTypeFormatterTests$1',
  'functional': 'Formatter',
  'technical': 'web'}]

In [11]:
query="""
UNWIND {domaininfo} AS d
MERGE (s:Domain:Business { name: d.business })
MERGE (t:Domain:Technical { name: d.technical })
MERGE (u:Domain:Functional { name: d.functional })
WITH s, t, u, d
MATCH (n:Type { fqn: d.fqn})
MERGE (n)-[:BELONGS_TO]->(s)
MERGE (n)-[:BELONGS_TO]->(t)
MERGE (n)-[:BELONGS_TO]->(u)
RETURN n.fqn, s.name, t.name, u.name
"""

json_data = graph.run(query, domaininfo=domaininfo).data()

In [14]:
import pandas as pd
pd.DataFrame(json_data).head()


Out[14]:
n.fqn s.name t.name u.name
0 org.springframework.samples.petclinic.Petclini... Pet petclinic Initializer
1 org.springframework.samples.petclinic.reposito... Pet jpa Repository
2 org.springframework.samples.petclinic.web.Visi... Visit web Controller
3 org.springframework.samples.petclinic.reposito... Vet jdbc Repository
4 org.springframework.samples.petclinic.web.Cras... Framework web Controller

In [16]:
import py2neo
graph = py2neo.Graph()
query="""
MATCH (d1:Business:Domain)<-[]-(n1:Type)-[*0..1]->(n2:Type)-[]->(d2:Business:Domain)
WHERE n2.fqn =~ "{}"
RETURN DISTINCT d1.name as name, 0 as lines, COLLECT(DISTINCT d2.name) as imports
""".format(pattern)

json_data = graph.run(query).data()
import json
with open ( "vis/flare-imports.json", mode='w') as json_file:
    json_file.write(json.dumps(json_data, indent=3))
    
json_data[:1]


Out[16]:
[{'imports': ['Owner', 'Framework', 'Pet', 'Clinic'],
  'lines': 0,
  'name': 'Owner'}]